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Method and system for stabilizing a video sequence 



FIELD OF THE INVENTION 

The present invention relates to a method for stabilizing a video recording of a 
made by a video camera, and to a stabilizing system for carrying out said method. 



scene : 



5 BACKGROUND OF THE INVENTION 

Miniaturization of video cameras in the recent years and inclusion of powerful 
zooms have led in these devices to an image stability problem, owing to the movement of the 
handheld video camera. Digital image stabilizing (DIS) systems have then been developed in 
order to remove this unwanted camera movement - or jitter - and thus to produce a sequence 
1 0 displaying requisite camera movement only and providing a more pleasant viewing 

experience. Jitter, defined as all the undesired positional fluctuations (translation, rotation, 
etc. .) of the image that are added to the intentional motion of the camera, can have a lot of 
significant adverse effects (for example, on motion estimation coding, since it may increase 
the magnitude or the number of the motion vectors and therefore a decrease in the run length 
15 of encoded motion vectors). In most DIS systems, a motion estimation is performed between 
two successive frames (at times t-1 and t) in order to get the global motion of the image 
frame. Said global motion is represented by a single vector called the global motion vector 
GMV(t). The problem is that the successive values GMV(t) contain both the intentional 
motion and the unintentional one (jitter). 
20 The document "Fast digital image stabilizer based on Gray-coded bit-plane 

matching", IEEE Transactions on Consumer Electronics, vol.45, n°3, August 1999, pp.598- 
603, describes a motion correction system allowing to decide whether the motion is 
intentional or not. To that end, the global motion vector is integrated with a damping 
coefficient, and the integrated motion vector IMV(t) thus defined designates the final motion 
25 vector correction to be applied to the current raw frame for motion correction. This integrated 
motion vector, provided for constructing a stabilized sequence, is given by the 

expression (1) : 

IMV(t) = a.IMV(t-l) + GMV(t) (1) 
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where GMV(t) is the global motion vector for the current frame considered at the time t and a 
is a damping factor (0 < a < 1) playing a role according to which the integrated motion vector 
converges to zero when there is no camera motion. The choice of this damping factor 
depends on the degree of stabilization desired, but it is crucial considering the type of camera 
motion, and the use of a constant value (for example) is often sub-optimal. 

In fact, separating the intentional motion from jitter over the course of a 
sequence is always a complex problem. For a real-time application, the main constraint is 
moreover that no delay is allowed, i.e. it is not possible to store some frames in memory and 
to apply the motion correction later, once the movement can be better known. All the 
solutions proposed to that end in the literature have more or less the same drawback : the 
jitter is attenuated - but not cancelled - in portions of sequences that have a low amplitude 
jitter and fixed position, but as soon as there is a panoramic motion or any other type of 
strong camera motion, the output filtered sequence hardly follows the real motion or follows 
it after a long delay. Moreover, the sampling window used for filtering causes a delay that is 
noticeable when correcting low amplitude jitter. There is consequently always to find a trade- 
off between the strength for attenuating low amplitude jitter and the ability to follow 
intentional motion with a delay as low as possible. 

SUMMARY OF THE INVENTION 

It is therefore an object of the invention to propose a filtering method allowing 
to perform an improved motion correction. 

To that end, the invention relates to a method for stabilizing a video recording 
of a scene made by a video camera and represented by video data, said method comprising 
the steps of : 

- subdividing said video data into a plurality of successive frames ; 

- dividing each of said successive frames into a plurality of blocks ; 

- determining for each block of each frame a motion vector representing the 
direction and magnitude of the motion in said block, said vector GMV at an instant t being 
called global motion vector GMV(t) and representing said motion at the instant t with respect 
to the previous frame ; 

- defining a modified vector, called integrated motion vector IMV(t) at the 
instant t and designating the final motion vector correction to be applied to the current frame 
in view of its motion correction, said integrated motion vector being given by the expression : 

IMV(t) = GMV(t) + a(.) . IMV(t - 1) 
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where GMV(t) is the global motion vector of the current frame at the instant t, a(.) is a 
variable adaptive factor and IMV(t - 1) is the integrated motion vector corresponding to the 

previous current frame ; and 

- modifying the video data according to the modified integrated motion vectors 

defined for each successive current frame. 

Preferably, this variable damping factor is calculated independently for the 
horizontal and vertical coordinates of the IMVs and GMVs, and thus the filtering along the 
two axes is always independent from each other. 

It may also be checked if the IMV correction is not above a given threhsold, 
corresponding to the extra input area authorized : if so, the correction is modified in order to 
stay within a predetermined allowed range, +/- 16 pixels for example. 

DETAILED DESCRIPTION OF THE INVENTION 

In accordance with the present invention, a stabilization method is proposed in 
which the proposed correction is now equal to the integrated motion vector IMV(t) 
previously defined, but with a damping coefficient which is now adaptive : 

IMV(t) = a(t, t-1, . . .) . IMV(t - 1) + GMV(t) (2) 
In this expression (2), the damping factor a(.) is now no longer a constant value, but an 
adaptive one depending for instance on the sum of the two last global motion vectors. This 
allows to track the beginning of intentional motion (the last global motion vector being 
usually too instable to be used per se). 

The correspondence table between the damping factor a(.) and the sum 
(GMV(t) + GMV(t-l)) of the two last global motion vectores is built in the following way : 

(a) a low sum of GMV values implies a high damping factor value, which 
strongly stabilizes the sequence, as if assuming static intentional camera ; 

(b) a high sum of GMV values implies a low damping factor value, and the 
sequence then follows more precisely the original motion. 

Some additional features may be proposed. It is for example advantageous if 
one has a filtering on the x-vector independent of y-vector filtering. Moreover, an out of 
bounds detection may be provided : said detection allows to check if the IMV correction is 
not above a given threshold, corresponding to an extra input area authorized, and to correct it 
to stay within an allowed range, for instance +/- 16 picture elements (pixels). 

The proposed stabilizing method thus creates long stabilized shots when jitter 
is centered around a certain spatial position, while following intentional motion when the 
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motion is too high. Some fast moving transitions may be stili visible, but they generally 
correspond to abrupt motion in the original sequence that must be followed in some wa> 
one wants to stay within the allowed extra image bounds. The stabilizing method is also 
effective in tracking slow moving motion. 
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CLAIMS: 



1 A method for stabilizing a video recording of a scene made by a video camera 

and represented by video data, said method comprising the steps of : 

- subdividing said video into a plurality of successive frames ; 

- dividing each of said successive frames into a plurality of blocks ; 

- determining for each block of each frame a motion vector representing the 
direction and magnitude of the motion in said block, said vector GMV at an instant t being 
called global motion vector GMV(t) and representing said motion at the instant t with respect 

5 to the previous frame ; 

- defining a modified vector, called integrated motion vector IMV(t) at the 
instant t and designating the final motion vector correction to be applied to the current frame 
in view of its motion correction, said integrated motion vector being given by the expression : 

IMV(t) = GMV(t) + a(.) . IMV(t - 1) 
10 where GMV(t) is the global motion vector of the current frame at the instant t, a(.) is a 
variable adaptive factor and IMV(t - 1) is the integrated motion vector corresponding to the 

previous current frame ; and 

- modifying the video data according to the modified integrated motion vectors 

defined for each successive current frame. 
15 2 . A stabilizing method according to claim 1, in which the variable damping 

factor a(.) is determined independently for the horizontal and vertical coordinates of the 
vectors. 

3 a stabilizing method according to anyone of claims 1 and 2, comprising an 

additional correction step, provided for checking if the correction of motion vector is not 
20 above a given threshold and, if yes, modifying said correction so that it stays within a 
predetermined allowed range. 

4. A system for stabilizing a video recording of a scene made by a video camera 

and represented by video data, said system comprising : 

- a frame storage for storing a plurality of successive frames of video data of 

25 the video recording ; 
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- a processor coupled to said frame storage for dividing each frame into a 
plurality of blocks, determining for each block of each frame a motion vector which 
represents the direction and magnitude of the motion in said block, said vector at an instant t 
being called global motion vector GMV(t) and representing said motion at the instant t with 
respect to the previous frame, defining a motion vector, called integrated motion vector 
IMV(t) at the instant t and designating the fmal motion vector correction to be applied to the 
current frame in view of its motion correction, said integrated motion vector being given by 
the expression : 

IMV(t) = GMV(t) + a(.) . IMV(t - 1) 
where GMV(t) is the global motion vector of the current frame at the instant t, a(.) is a 
variable adaptive factor and IMV(t - 1) is the integrated motion vector corresponding to the 
previous current frame, and modifying the video data according to the modified integrated 
motion vectors defined for each successive current frame. 
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Abstract 

The invention relates to a method for stabilizing a video recording of a scene 
made by a video camera and represented by video data, said method comprising the steps of 
subdividing said video data into a plurality of successive frames themselves divided into a 
plurality of blocks, determining for each block of each frame a motion vector representing 
the direction and magnitude of the motion in said block, said vector at an instant t being 
called global motion vector GMV(t) and representing said motion at the instant t with respect 
to the previous frame, defining a modified vector called integrated motion vector IMV(t) at 
the instant t and designating the final motion vector correction to be applied to the current 
frame in view of its motion correction, said integrated motion vector being given by the 
expression : 

IMV(t) - GMV(t) + a(.) . lMV(t - 1) 
where aQ is a variable adaptive factor and IMV(t - 1) is the integrated motion vector 
corresponding to the previous current frame, and modifying the video data according to the 
modified integrated motion vectors defined for each successive current frame. 
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